In this discussion of prior art, reference will be made to the following articles:
1. Holbrook A, Rolnick M L, and Bailey C W. Treatment of vocal abuse disorders using a vocal intensity controller. Journal of Speech and Hearing Disorders, 39:298-303, 1974.
2. Davis S M and Drichta C E. Biofeedback Theory and Application in Allied Health. Biofeedback and Self-Regulation, Vol. 5, No. 2, 1980.
3. Brody D M, Nelson B A, and Brody J F. The use of visual feedback in establishing normal vocal intensity in two mildly retarded adults. Journal of speech and hearing disorders, 40:502 507, 1975.
4. Roll D L. Modification of nasal resonance in cleft palate children by informative feedback. Journal of Applied Behavior Analysis, 6:397-403, 1973.
5. Stark R E. The use of real-time visual displays of speech in the training of a profoundly deaf nonspeaking child: A case report. Journal of Speech and Hearing Disorders, 36:397-409, 1971.
6. Guitar B. Reduction of stuttering frequency using analog electromyographic feedback. Journal of Speech and Hearing Research, 18:672-685, 1975.
7. Hison T J. Respiratory function in speech. In F D Minifie, T J Hison and F Williams (Eds), Normal Aspects of Speech, Hearing and Language, Prentice-Hall, Englewood Cliffs, N.J., 1973.
8. Stassen H H, Bomben G, Gunther E. Speech characteristics in depression. Psychopathology, 24:88-105, 1991.
9. Murray I R, Arnott J L. Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion. Journal of Acoustical Society of America, 93:1097 1108, 1993.
10. Brenner M, Dohersy T, Shipp T. Speech measures indicating workload demand, Aviation Space and Environmental Medicine, 65:21 26, 1994.
Biofeedback may be defined as the technique of using equipment (usually electronic) to reveal to human beings some of their internal physiological events, normal and abnormal, in the form of visual and auditory signals in order to teach them to manipulate these otherwise involuntary or unfelt events (such as heart beat and emotions) by manipulating the displayed signals. This technique allows an open feedback loop to be closed by a person's volition so as to modify the outcome based on preset goals. To achieve such goals requires voluntary cooperation on the part of the subject. Psychologically, the functions to be controlled are associated with the structures that determine the emotional status of the organism.
Various types of devices have been developed and used in biofeedback systems in which one or more signals representative of physiological variables are fed back. These variables constitute, for example, muscle activity, galvanic skin resistance, heart rate, temperature or blood pressure. In the prior art, appropriate transducers have been used for transforming these signals to visual or audio stimuli. The subject is expected to control mentally one or more of the monitored physiological functions, thus modifying the outcome of the physiological variables.
Emotional feeling is considered basically a perception of bodily changes and reflected through measurements in the autonomic changes. Some of the measurements are considered to indicate generalized stress or arousal rather than particular emotions. It is known to use biofeedback for training subjects in relaxation techniques by measuring stress-related variables such as pulse rate, breathing pattern, blood pressure, temperature and the electrical resistance (galvanic skin resistance) of the palm of the hand or its fingers.
The response of an individual to a verbal provocation which touches him personally is usually related to an emotional reaction with some degree of stress. If, for example, a person tells a lie, this will result in a specific physiological reaction whose comparison to the complementary response when no lie is told may be indicative of a lie having been told. However, whilst a particular emotional state related to stress will always produce a similar physiological change, it is not possible to infer the cause of such stress merely from the fact of such a physiological change. Thus, prior art in the field of biofeedback which aims to manipulate an emotional state, is confined mainly to the aspects of arousal and relaxation.
Spector (U.S. Pat. No. 5,209,494) discloses a biofeedback system which monitors an involuntary physiological function of an individual and indicate the individual's-state of stress, making it possible for the individual to exercise control over the function being monitored. Stress is measured using a temperature sensor device.
Agoston (U.S. Pat. No. 4,184,485) discloses a measuring arrangement for decreasing the emotional influence on instrumental diagnostic measurements using heart rate in a biofeedback system in which the change of a subject's emotional state is indicated by changing the tone pitch of an audio output heard by the subject or by showing the pulse rate on a digital readout indicator.
Shiga (U.S. Pat. No. 4,345,505) discloses a self-training biofeedback system in which the electrical activity of a subject's brain is used to indicate his state of relaxation, and a binary count output system is employed for indicating the relaxation period.
Bittman (U.S. Pat. No. 5,343,871) teaches the use of an apparatus for mediating a biofeedback session with a human subject in which the clarity of an image and sound improve as an indication of success the subject in reaching a state of relaxation.
Dardik (U.S. Pat. No. 5,163,439) teaches the use of biofeedback for enabling a subject to control his pulse rate, thus learning to relax and thereby reduce tension and its physiological consequences.
The applications of biofeedback techniques using speech as the physiological variable to be fed back and controlled has been used in the area of speech pathologies in controlling vocal intensity, resonance, and pitch. However, it has been used very little in teaching patients with hearing deficits to speak with proper articulation and expressing emotion. Speech is an overlaid function: there is no specific organ for speech. Instead, anatomical structures of the aerodigestive tract must function in a coordinated manner to produce intelligible sound. Normal speech production requires the coordinated activity of the respiratory muscles and those muscles responsible for phonetics-and articulation. A controlled delivery of air in expiration is needed to allow the muscles of the larynx and oral pharynx to modulate the vibrations we interpret as voice. This is a process that necessarily involves the precise control of muscular functions. As a result, any aberrant muscle activity would naturally cause, or contribute to, many types of speech modifications.
It is well known that emotion and stress (a psychological state which is accompanied by the specific emotions of anxiety, fear and/or anger) modify the speech so that the changes are recognizable by other people. Sentences as "you sound sad" or "you sound angry" are clearly indicative that a person's voice reflect his emotional state. Among the speech characteristics, the fundamental frequency (F.sub.0) is an important variable that changes when there is an emotional change and varies between different emotions. Several others speech characteristics are considered to be important in the analysis of emotion from speech (Stassen et al..sup.8). Such characteristics include speech flow (the speed at which utterances are produced as well as the number and duration of temporary breaks in speaking); loudness (the amount of energy used to articulate utterances, and the speaker's dynamic expressiveness); intonation (the manner of producing utterances with respect to rise and fall in pitch); mean utterance duration and variability of utterance duration; mean pause duration and variability of pause duration.
Murray et al..sup.9 reviewed the state of the art in our understanding of human vocal emotion. The acoustic properties appearing to be among the most sensitive indicators of emotion were attributes that specified the contours of F.sub.0 throughout an utterance. Murray et al..sup.9 refer to a multi-variable model in which different speech characteristics are associated with emotions such as anger, happiness, sadness, fear, and disgust. For example, the emotion of anger is expressed in faster speech rate, higher pitch average, wider pitch, higher intensity, abrupt pitch changes and tense articulation.
Brenner et al..sup.10 show that stress in speech can be detected using speech rate, pitch, vocal intensity and derived speech measure (z-scores).
Holbrook et al..sup.1 discloses an instrument for controlling voice intensity in a treatment program for patients with dysphonia (i.e. roughness of sound) related to vocal cord lesion and to laryngeal hypertension. The instrument provides auditory feedback contingent on excessive vocal intensity.
As reported by Davis et al..sup.2, Brody et el..sup.3 use a vocal-activated relay to provide visual feedback of vocal amplitude for subjects who habitually used very soft voices. Subjects demonstrate significant increases in their use of normal voice intensity.
As reported by Davis et al..sup.2, Roll.sup.4 utilizes a biofeedback approach in patients who suffer from hypernasal speech (an excessive undesirable amount of perceived nasal cavity resonance that occurs during the phonation of vowels). The resonance characteristics of vowels sounds are treated as operant behaviors. Differential feedback is arranged for nasal versus non-nasal responses, so as to display a visual indication when the nasal vibration exceeds an arbitrary unit, thereby teaching patients to control their nasal vibration.
Stark.sup.5 reports the use of real-time amplitude contours and spectral display of speech in the training of speech-production skills.
The development of electromyographic (EMG) biofeedback as a means of measuring, recording, and displaying the electrical activity of living muscle has significant implications for the assessment and treatment of communication disorders. EMG recording provides a more objective means of measuring and characterizing the nature of muscle activity during speech, typically focused on few muscle groups.
Most prior studies limit their observation to a single level of the speech mechanism such as the laryngeal (in stuttering and dysarthria as done by Guitar.sup.6) or respiratory muscles (Hison.sup.7) both reported by Davis et al..sup.2. The use of biofeedback in teaching communication disorders introduces a quantitative measure to the improvement of the speech. However, qualities of speech such as emotion have not been treated by biofeedback.
It is well known that when subjects are tired, speech characteristics are affected. Speech in a tired subject has a lower speech flow, a slower pitch, a lower intensity, a lower derived speech measure, a lower intonation, a larger mean pause duration and a larger variability of mean pause duration. Nevertheless, speech fatigue has not been proposed either as a criterion for finding a subject's fatigue level so as to determine his or her mental state, or as a means to avoid treating subjects who are tired.